IN THE SPECIFICATION 
Please replace the paragraph at page 1, lines 10-15, with the following rewritten 
paragraph: 

There is a system in which a user 84 is located in front of a display 82 having cameras 
81-1 and 81-2, shown in Fig. [[28]] 1, and a mouse [[etc.]] and the like is operated by finger- 
pointing of the user 84, in which the position of the user 84 is determined and the mouse 
[[etc.]] and the like is operated from the images of as small few as two or three cameras. 

Please replace the paragraph at page 1, line 19 to page 2, line 2, with the following 
rewritten paragraph: 

Furthermore, also the hand-sign recognition system, as shown in Fig. [[29]] 2, which 
is used as an interface apparatus that acquires and recognizes the image of a hand alone of a 
user 94 in a fixed environment in front of a camera 91, has the same problem. In the 
drawing, numeral 92 denotes a display and numeral 93 indicates a recognition object region. 

Please replace the paragraph at page 4, line 13 to page 6, line 3, with the following 
rewritten paragraph: 

[5] In the interface apparatus as set forth in [1], the positional-posture and arm- 
pointing recognition means for extracting arm pointing recognizes a pointing action in such a 
way that: the recognition means divides three-dimensional distance information obtained 
from the stereo cameras into levels by 20 cm according to the indoor coordinate system by a 
different-level extracting method; projects a dot sequence in each level onto a two- 
dimensional plane and then binarizes it into a two-dimensional binary image; labels the 
images on a level-to-level basis; determines the overall center of gravity of the clusters; 
stacks the center of gravity determined in each two-dimensional plane in levels on object- to- 
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object basis again to use as a three-dimensional dot sequence; plots the center of gravity of 
each level along the Z-axis, in which eight levels (the upper half of a body) from the 
uppermost level (the head) are plotted on the X-Y plane; wherein when there is a large outlier 
compared to the overall center of gravity , determines that an arm-pointing action has been 
made; determines the direction of the body by calculating the image moments of the binary 
images of the eight levels from the uppermost, the image moment being a rectangle 
equivalent to the binary image, and determining the vertical direction of the long side of a 
level having an area within a specified range and in which the difference between the long 
side and the short side of the image moment is the largest of the acquired eight sets of data as 
the direction of the body; when the arm-pointing action has been recognized, the recognition 
means determines the direction of the arm pointing in such a way that: it determines the 
direction of the arm pointing on the X-Y plane by drawing a perpendicular bisector between 
the overall center of gravity and the center of gravity of the binary image of a level of which 
the center of gravity is farthest from the overall center of gravity, erasing the binary image in 
the region corresponding to the body of the person to leave only the image of the arm; 
calculates the image moment of the binary image of only the arm to d e t e rmain a determine the 
long side, the center of gravity, the position of the distal end of the arm, and the overall center 
of gravity; and determines the Z-direction of the arm pointing from the stature, the height of 
eyes, and arm-length coefficient. 

Please replace the paragraph at page 7, lines 15-18, with the following rewritten 
paragraph: 

[14] In the interface apparatus as set forth in [5], when the average center of gravity 
of a partner is found within a specified radius [[r]] around the distal end of the user's arm 
pointing, it is determined not to be arm pointing. 
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Please insert the following new paragraphs at page 8, between lines 23 and 24: 

Fig. 1 is a diagram of a related art; 

Fig. 2 is a diagram of another related art; 

Please replace the paragraph at page 8, line 24 to page 9, line 1, with the following 
rewritten paragraph: 

Fig. [[1] 3 is a diagram showing the configuration of a system according to an 
embodiment of the invention; 

Please replace the paragraph at page 9, lines 2-3, with the following rewritten 
paragraph: 

Fig. [[2]] 4 is a diagram showing the configuration of a system according to an 
embodiment of the invention; 

Please replace the paragraph at page 9, lines 4-5, with the following rewritten 
paragraph: 

Fig. [[3]] 5 is a block diagram of an integrated-information recognition system shown 
in Figs. [[l]]3and [[2]] 4; 

Please replace the paragraph at page 9, lines 6-7, with the following rewritten 
paragraph: 

Fig- [[4]] 6 is a flowchart for the operation of the embodiment of the invention; 
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Please replace the paragraph at page 9, lines 8-9, with the following rewritten 
paragraph: 

Fig- [[5]] 7 is an explanatory diagram of a different-level extracting method used in 
the invention; 

Please replace the paragraph at page 9, lines 10-11, with the following rewritten 
paragraph: 

Fig. [[6]] 8 is an explanatory diagram (1) of a posture-recognition processing method 
used in the invention; 

Please replace the paragraph at page 9, lines 12-13, with the following rewritten 
paragraph: 

Fig. [[7]] 9 is an explanatory diagram (2) of the posture-recognition processing 
method used in the invention; 

Please replace the paragraph at page 9, lines 14-15, with the following rewritten 
paragraph: 

Fig. 8 is an Figs. 10(a)- 10(d) are explanatory diagram diagrams (3) of the posture- 
recognition processing method used in the invention; 

Please replace the paragraph at page 9, lines 16-17, with the following rewritten 
paragraph: 

Fig- [[9]] ii is an explanatory diagram (4) of the posture-recognition processing 
method used in the invention; 
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Please replace the paragraph at page 9, lines 18-19, with the following rewritten 
paragraph: 

Fig. [[10]] 12 is an explanatory diagram (arm-pointing diagram 1) for the operation of 
an embodiment of the invention; 

Please replace the paragraph at page 9, lines 20-21, with the following rewritten 
paragraph: 

Fig. [[1 1] 13 is an explanatory diagram (arm-pointing diagram 2) for the operation of 
an embodiment of the invention; 

Please replace the paragraph at page 9, lines 22-24, with the following rewritten 
paragraph: 

Fig. [[12]] 14 is an explanatory diagram (a case in which a leg is stretched out) for the 
operation of an embodiment of the invention; 

Please replace the paragraph at page 9, line 25 to page 10, line 1, with the following 
rewritten paragraph: 

Fig. 13 is an Figs. 15(a), 15(b) are explanatory diagram diagrams (slight arm pointing 
1) for the operation of an embodiment of the invention; 

Please replace the paragraph at page 10, lines 2-3, with the following rewritten 
paragraph: 

Fig. [[14]] 16 is an explanatory diagram (slight arm pointing 2) for the operation of an 
embodiment of the invention; 
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Please replace the paragraph at page 10, lines 4-6, with the following rewritten 
paragraph: 

Fig. 15 is an Figs. 17(a), 17(b) are explanatory diagram diagrams (case 1 in which 
both arms are spread out) for the operation of an embodiment of the invention; 

Please replace the paragraph at page 10, lines 7-9, with the following rewritten 
paragraph: 

Fig. [[16]] 18 is an explanatory diagram (case 2 in which both arms are spread out) 
for the operation of an embodiment of the invention; 

Please replace the paragraph at page 10, lines 10-12, with the following rewritten 
paragraph: 

Fig. 17 is an Figs. 19(aV 19(b) are explanatory diagram diagrams (a case in which a 
partner stands by such as shaking hands) for the operation of an embodiment of the invention; 

Please replace the paragraph at page 10, lines 13-14, with the following rewritten 
paragraph: 

Fig. [[18]] 20 is a flowchart for the identification of actions of an embodiment of the 
invention; 

Please replace the paragraph at page 10, lines 15-17, with the following rewritten 
paragraph: 

Fig- [[19]] 21 is a flowchart for details of part of the operation of the arm-pointing 
recognition section in Fig. [[18]] 20 for a lying posture; 
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Please replace the paragraph at page 10, lines 18-20, with the following rewritten 
paragraph: 

Fig. [[20]] 22 is a flowchart for details of part of the operation of the arm-pointing 
recognition section in a specified area in the invention; 

Please replace the paragraph at page 10, lines 21-22, with the following rewritten 
paragraph: 

Fig. [[21]] 23 is a diagram of the images of the base of a bed for calculating the rising 
angle of the bed; 

Please replace the paragraph at page 10, lines 23-24, with the following rewritten 
paragraph: 

Fig. [[22]] 24 is a flowchart for the registration of the operating-unit database of the 
invention; 

Please replace the paragraph at page 10, line 25 to page 11, line 1, with the following 
rewritten paragraph: 

Fig. 23 is an Figs. 25(a)-25(c) are explanatory diagram diagrams of three basic 
patterns of a pointing action of the invention; 

Please replace the paragraph at page 11, lines 2-3, with the following rewritten 
paragraph: 

Fig. [[24]] 26 is a flowchart for a crossing operation of the invention; 



8 



Please replace the paragraph at page 11, lines 4-5, with the following rewritten 
paragraph: 

Fig. [[25]] 27 is a flowchart for a panel touch operation of the invention; 

Please replace the paragraph at page 11, lines 6-7, with the following rewritten 
paragraph: 

Fig. [[26]] 28 is a flowchart for a panel gesture operation of the invention; and 

Please replace the paragraph at page 1 1, lines 8-9, with the following rewritten 
paragraph: 

Fig. [[27]] 29 is an explanatory diagram for the operation of an embodiment of the 
invention[[;]]. 

Please delete the paragraphs at page 1 1, lines 10-11: 
Fig. 28 is a diagram of a related art; and 
Fig. 29 is a diagram of anoth e r r e lat e d art. 

Please replace the paragraph at page 11, lines 15-18, with the following rewritten 
paragraph: 

Figs- [[1]] 2 an d [[2]] 4 are diagrams showing the configuration of a system according 
to an embodiment of the invention. Fig. [[3]] 5 is a block diagram of the integrated- 
information recognition system shown in Figs. [[1]] 3 and [[2]] 4. 
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Please replace the paragraph at page 11, line 19 to page 12, line 4, with the following 
rewritten paragraph: 

Referring to Fig. [[1]] 3, an indoor space 5 is photographed by a plurality of stereo 
cameras 1-1 to 1-n so as to leave no blind spot, and users 4-1, 4-2 to 4-n move freely in the 
indoor space 5. Two or more image-pickup devices are fixed to each of the stereo cameras 1 
in parallel. The image pickup outputs from the stereo cameras 1-1 to 1-n are delivered to an 
image processor 2. The stereo cameras 1 themselves are of a known type, such as Digiclops 
produced by Point Grey Research Inc., or Acadia produced by David Sarnoff Research 
Center. 

Please replace the paragraph at page 12, lines 12-19, with the following rewritten 
paragraph: 

Fig- [[1]] 2 shows an example in which all the indoor space is covered as a 
recognition object region (operable space) 5; in Fig. [[2]] 4, only one of the stereo cameras 1 
is placed downward and only the indoor space thereunder is shown as the recognition object 
region (operable space) 5. While in the operable space 5 of Fig. [[2]] 4 only one user 4 is 
shown as an object, the fundamental system is similar to that shown in Fig. [[1]] 3. 

Please replace the paragraph at page 12, line 20 to page 13, line 17, with the following 
rewritten paragraph: 

Referring to Fig. [[3]] 5, the integrated-information recognition system 3 includes a 
positional-posture recognition section 31, an arm-pointing recognition section 32, a pointing- 
action recognition section 33, an operating-unit database 34, and an operating-unit 
registration section 35. The positional-posture recognition section 31 extracts distance data 
from three-dimensional information collected by the image processor 2 along the indoor- 
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space coordinate system (the X-, Y-, Z axis Z-axes in Figs. [[1]] 3 and [[2]] 4) by the 
different-level extracting method shown in Fig. [[5]] 7 and projects the data onto two- 
dimensional planes, thereby performing posture recognition process for multiple users. The 
arm-pointing recognition section 32 identifies the direction of the arm pointing from the 
obtained two-dimensional projection drawings of the multiple users. The pointing-action 
recognition section 33 recognizes the pointing action from the obtained time-series data of 
the arm-pointing direction. The operating-unit database 34 stores the information on the 
operation object unit 6 that is an object of arm-pointing operation and an operation method. 
The operating-unit registration section 35 stores the basic information, position, and 
operation method of the operation object unit 6. 

Please replace the paragraph at page 13, line 18 to page 14, line 3, with the following 
rewritten paragraph: 

Referring to Figs. [[1]] 3, [[2]] 4, and [[3]] 5, an operation-command transmitter- 
receiver 9 operates the operation object unit 6 by transmitting an operation command to the 
operation object unit 6 determined by the integrated-information recognition system 3 and 
receiving it with the operation object unit 6. At the same time, the operation-command 
transmitter-receiver 9 receives necessary information from the operation object unit 6. The 
method for transmitting and receiving the operation command includes communication 
methods using weak electric waves having no directivity, such as Bluetooth and ECHONET. 

Please replace the paragraph at page 14, lines 4-9, with the following rewritten 
paragraph: 

While Figs. [[1]] 3 and [[2]] 4 show the embodiment of the interface apparatus in a 
room, the invention is not limited to this embodiment. For example, the invention can also be 
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applied to any case in which a pointing action is used, for example, in factories or public 
space, in place of the indoor space. 

Please replace the paragraph at page 14, lines 10-12, with the following rewritten 
paragraph: 

Fig- [t 4 ]] 6 is a flowchart for the operation of the embodiment of the invention, 
describing part of Fig. [[3]] 5 in detail. 

Please replace the paragraph at page 14, lines 13-19, with the following rewritten 
paragraph: 

All the users 4-1 to 4-n who go in and out of the indoor space 5 of Fig. [[1]] 3 are 
photographed by the stereo cameras 1-1 to 1-n, and the image-pickup outputs are delivered to 
the image processor 2. That is, the behaviors of persons who go in and out of this indoor 
space 5 are all photographed and monitored after entering the room until going out, according 
to the flow of Fig. [[4]] 6. 

Please replace the paragraph at page 14, line 20 to page 15, line 1, with the following 
rewritten paragraph: 

The image processor 2 produces distance information based on the color images 
within the visual field in units of camera and the coordinate system of the room, and delivers 
the information to the integrated-information recognition system 3. The coordinate system X, 
Y, and Z (Figs. [[1]] 3 and [[2]] 4) common to all the stereo cameras 1-1 to 1-n is preset. 
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Please replace the paragraph at page 15, lines 5-14, with the following rewritten 
paragraph: 

The three-dimensional distance information (X 5 Y, Z) acquired from the stereo 
cameras 1-1 to 1-n in step S41 is not operated as a whole but is divided into levels by 20 cm 
according to the indoor coordinate system to cut out three-dimensional distance dot 
sequences, as shown in Fig. [[5]] 7. To prevent information from being lost in the boundary, 
the information is cut again with a displacement of 10 cm and as such, the continuous three- 
dimensional distance dot sequences can be cut out without a loss in information (step S42). 

Please replace the paragraph at page 15, lines 15-23, with the following rewritten 
paragraph: 

Then, as shown in Fig. [[6]] 8, the dot sequence in each level (0 to 15 in Fig. [[5]] 7) 
is projected onto a two-dimensional plane and is then binarized. Thus, the information in 
each level is brought into a two-dimensional binary image. It is labeled on a level-to-level 
basis (two users are labeled as A and B in Fig. [[6]] 8) and thereafter the overall center of 
gravity of each cluster (the center * in "Total" of Fig. [[6]] 8) is determined (step S43). Thus 
multiple persons can be processed by being labeled initially. 

Please replace the paragraph at page 15, line 24 to page 16, line 2, with the following 
rewritten paragraph: 

Referring next to Fig. [[7]] 9, the centers of gravity determined in the two- 
dimensional planes are stacked again in levels for each of the labeled object and they are used 
as a three-dimensional dot sequence (step S44). 
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Please replace the paragraph at page 16, lines 3-16, with the following rewritten 
paragraph: 

Referring to Fig. [[8(a)]] 10(a) , the center of gravity of each level is plotted along the 
Z-axis (along the height), in which those of eight levels (corresponding to the upper half of a 
body) from the uppermost level (the head) are plotted on the X-Y plane, wherein when there 
is a large outlier, it is determined that an arm-pointing action has been made. To measure the 
direction of the body at the same time, the image moments of the binary images of eight 
levels from the uppermost (only those of six lev e ls are shown in Fig. 8(b) are calculated, as 
shown in Fig. [[8(b)]] 10(b) (only those of six levels are shown in Fig. 10(b)) . The image 
moment denotes a rectangle equivalent to the binary image, shown in Fig. [[9]] H, and the 
center of gravity of the rectangle (xg, yg), slope 0, long side LI, short side L2 are given by 
equations (1). 

Please replace the equation at the top of page 17 with the following new equation: 
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Please replace the paragraph at page 17, line 7 to page 18, line 2, with the following 
rewritten paragraph: 

When an arm-pointing action has been recognized (Yes in step S46), the direction of 
the arm pointing is determined [[by]] using the binary image of the level (Fig. [[8c]] 10(c) ) in 
which the center of gravity is farthest from the overall center of gravity. Specifically, to be 
described later in detail, a perpendicular bisector is drawn between the overall center of 
gravity and the center of gravity of the level corresponding to arm-pointing and the binary 
image in the region corresponding to the body of the person is erased (Fig. [[8(d)]] 10(d) ) to 
leave only the image of the arm. The image moment of the binary image of only the arm is 
calculated to determine the long side LI, short side L2, the center of gravity (xc, yc), and 
slope 0, thereby determining the direction on the X-Y plane. 

Please replace the paragraph at page 18, lines 3-1 1, with the following rewritten 
paragraph: 

The Z-direction indicative of the height direction includes no information on the 
vertical direction because the angle a of the arm acquired from the three-dimensional 
distance information is projected on the horizontal plane. Thus the angle ocV of the arm in 
the vertical direction is determined by using equation (2). 



where h is the stature, heye is the height of the eyes, harm is the height of the arm, and 1 is an 
arm length coefficient. 
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Please replace the paragraph at page 18, lines 12-23, with the following rewritten 
paragraph: 

Where the base of the direction is not the height of the shoulder joint but the height of 
the eyes, heye. This is because, as shown in Fig. [[10]] 1_2, it is often observed by actual 
observation that a person points an object with the arm in the direction of the line between the 
eyes and the tip of the finger, not in the direction 104 of the arm itself The height of eyes, 
heye 102, is obtained by subtracting 10 cm from the height of the head, 103 (stature h). The 
average of Japanese adults, 0.4, is used as the arm-length coefficient 1. In Fig. [[10]] 12 , 
numeral 101 denotes the height of the shoulder and numeral 105 indicates the direction of 
arm pointing. 

Please replace the paragraph at page 19, lines 1-11, with the following rewritten 
paragraph: 

Also the horizontal direction of arm pointing is determined from the positional 
relationship between the position of the eyes and the distal end of the arm. As shown in Fig. 
[[11]] 12, g is the position (xg, yg) of the average center 111 of gravity of a person, c is the 
position (xc, yc) of the center 1 12 of gravity of the arm, p is the position (xp, yp) of the distal 
end of the arml 13 arm 113 , ah is the horizontal direction 1 15 of the arm, and LI is the long 
side of the image moment (arm body) 1 14 obtained in Fig. [[8(d)]] 10(d) . Where the 
horizontal direction ah 1 15 of the arm pointing is obtained from equation (3). 

Please replace the paragraph at page 19, lines 13-14, with the following rewritten 
paragraph: 

Referring to Fig. [[11]] 13, numeral 116 indicates the direction of arm pointing. 
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Please replace the paragraph at page 20, lines 14-18, with the following rewritten 
paragraph: 

Fig. [[12]] L4 is a schematic diagram of the case in which the user stretches out his 
leg. Numeral 121 denotes a chair, 122 a leg stretched forwards, 123 a lower recognition limit 
of arm pointing, 124 the height of the head, and 125 the sitting height. 

Please replace the paragraph at page 20, line 19 to page 21, line 6, with the following 
rewritten paragraph: 

As shown in Fig. [[12]] 14, when the user sits in the chair 121 and stretches out the 
leg 122 forward, the three-dimensional point sequence of the stretched leg 122 resembles that 
of arm pointing, which may cause the system to sense arm pointing. To prevent it, restriction 
due to the shape of human body is used. In short, regardless of a person being in a standing 
or sitting position, it is obvious that the arm at arm pointing is located higher than the waist. 
Thus, the lower limit hmin 123 to the determination of arm pointing is obtained from the 
head height hhead 124 and the height hsit corresponding to the sitting height 125 by equation 
(4), and the detection of arm pointing at heights lower than that is ignored. 

Please replace the paragraph at page 21, lines 10-14, with the following rewritten 
paragraph: 

Fig. [[13]] 15 is a schematic diagram of slight arm pointing, Fig. [[13(a)]] 15(a) being 
a top view and Fig. [[13(b)]] 15(b) is a front view thereof. Numeral 131 denotes slight arm 
pointing, 132 an image moment, 133 the long side LI of the image moment 132, and 134 the 
lower limit of the long side LI. 
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Please replace the paragraph at page 21, lines 15-18, with the following rewritten 
paragraph: 

In the case of slight arm pointing 131 as shown in Fig. [[13]] 15, wherein when a 
predetermined ratio of the long side LI of the image moment 132 to the stature is not 
reached, it is determined not to be arm pointing. 

Please replace the paragraph at page 21, line 19 to page 22, line 2, with the following 
rewritten paragraph: 

Referring to Fig. [[14]] 16, when rl/r2 is less than a value set from the stature, where 
rl is the distance from an average center of gravity 141 to the distal end 144 of arm pointing 
and r2 is the distance from the average center of gravity 141 to the base end 142 of arm 
pointing, it is similarly determined not to be arm pointing. Numeral 143 indicates the center 
of gravity of the arm. 

Please replace the paragraph at page 22, lines 5-9, with the following rewritten 
paragraph: 

Fig. [[15]] 17 is a schematic diagram (1) in the case where both arms are spread out, 
Fig. [[15(a)]] 17(a) being a perspective view and Fig. [[15(b)]] 17(b) being a top view 
thereof. Numeral 151 denotes spread both arms and numeral 152 indicates the image 
moment when the both arms are spread out. 

Please replace the paragraph at page 22, lines 10-17, with the following rewritten 
paragraph: 

As shown in Fig. [[15]] 17, when both arms are spread out, the area S of the image 
moment 152 of the spread arms is obtained from the long side LI and the short side L2 by 
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equation (5). The upper limit is set for the area S and the lower limit is set for the long side 
LI, wherein when the area S has exceeded the upper limit, or the long side LI has fallen 
below the lower limit, it is determined not to be arm pointing and so ignored. 

Please replace the paragraph at page 22, lines 19-23, with the following rewritten 
paragraph: 

Fig. [[16]] 18 is a schematic diagram (2) in the case where both arms are spread out. 
Numeral 161 denotes an average center of gravity, 162 the base end of the arm pointing, 163 
the distal end of the arm pointing, and 164 the image moment thereof. 

Please replace the paragraph at page 23, lines 1-8, with the following rewritten 
paragraph: 

As shown in Fig. [[16]] 18, in the case of false arm pointing in which, referring to the 
image moment 164, the ratio of the distance r3 between the distal end 163 of the arm pointing 
and the average center of gravity 161 to the distance r4 between the base end 162 of the arm 
pointing and the average center of gravity 161 is greater than a set value, it is determined not 
to be arm pointing as in slight arm pointing. 

Please replace the paragraph at page 23, lines 11-17, with the following rewritten 
paragraph: 

Fi §- [[I 7 ]] 19 is a schematic diagram of the case in which a partner stands by the 
person such as when shaking hands, Fig. [[17(a)]] 19(a) being a top view and Fig. [[17(b)]] 
19 ( b ) being a perspective view thereof. Numeral 171 denotes the slight arm pointing of the 
partner, 172 the direction of the arm pointing, 173 the distal end of the arm pointing, and 174 
an average center of gravity of the partner. 



19 



Please replace the paragraph at page 23, lines 1 8-23, with the following rewritten 
paragraph: 

As shown in Fig. [[17]] 19, in the case where the partner is present in the immediate 
area of the person such as when shaking hands, wherein when the average center of gravity 
174 of the partner is found within a specified radius r around the distal end 173 of the arm 
pointing, it is determined not to be arm pointing. 

Please replace the paragraph at page 25, lines 10-1 1, with the following rewritten 
paragraph: 

Fig. [[18]] 20 is a flowchart for describing the distinction of actions. 

Please replace the paragraph at page 25, line 12 to page 26, line 13, with the following 
rewritten paragraph: 

A specified area in the indoor space 5 is first registered in advance (step S91). It is 
checked whether the user 4 is present in the specified area (step S92). When it is determined 
from time series information of the user 4 that is brought into three dimensions according to 
the procedure shown in Figs. [[5]] 7 to [[8]] 10 (Yes in step S92), recognition of arm pointing 
for the specified area is made in step S97. When the user 4 is present outside the specified 
area (No in step S92), information on the height of the slices is determined to check whether 
the user 4 is in a standing or sitting posture (step S93). When the user 4 is in a standing or 
sitting posture (Yes in step S93), general arm-pointing recognition for the standing or sitting 
posture is performed in step S96. When the user 4 is in neither the standing nor sitting 
posture (No in step S93), it is determined in step S94 whether the user 4 is in a lying posture. 
When it is determined in step S94 from the range of the distance images to be in a lying 
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posture (Yes in step S94), recognition of arm pointing for the lying posture is made in step 
S95. The steps S95 to S97 correspond to the operation of the arm-pointing recognition 
section 32. When it is determined that the user 4 is outside the specified area and is not in 
any of the standing, sitting, and lying-down postures (No in step S94), the procedure returns 
to step S93 to continue the recognition of the posture. 

Please replace the paragraph at page 26, lines 14-16, with the following rewritten 
paragraph: 

Fig. [[19]] 2! is a flowchart for the details of part of the operation of the arm-pointing 
recognition section 32 for a lying posture. 

Please replace the paragraph at page 27, lines 11-16, with the following rewritten 
paragraph: 

Fig. [[20]] 22 is a flowchart for the details of part of the operation of the arm-pointing 
recognition section 32 in a specified area. Here the object is a nursing bed (reclining bed), on 
which recognition of arm pointing can be made in a typical sitting or lying posture 
irrespective of the use of a reclining function. 

Please replace the paragraph at page 27, lines 17-25, with the following rewritten 
paragraph: 

With the position of the nursing bed being set as the specified area, the position, size, 
height, and bending position of the bed in the indoor space 5 are registered in the operating- 
unit database 34 in advance in step SI 101. In step SI 102, the rising angle of the bed is 
calculated from the slice image of the base of the bed. As shown in Fig. [[21]] 23, the rising 



21 



angle of the bed can be determined from the size of the bed and the size of the moving part 
(the whitish part in the photographs) in the image of the base of the bed. 

Please replace the paragraph at page 28, lines 6-14, with the following rewritten 
paragraph: 

When the rising angle is large, the recognition of arm pointing particular to a nursing 
bed is continued. In step SI 105, a provisional center of gravity of a user 4 is determined 
from the entire distance image on the plane of the bed except the backrest. When it is 
determined in step SI 106 that the provisional center of gravity of the user 4 is in a position 
closer to the head than the bed bending position, as shown in Fig. [[21]] 23, it is determined 
that the user 4 leans on the bed (step SI 107). 

Please replace the paragraph at page 29, lines 13-24, with the following rewritten 
paragraph: 

Fig. [[23]] 25 shows three basic patterns of a pointing action. In the case in which the 
state of arm pointing is recognized accurately by positional-posture recognition section and 
arm-pointing recognition section, wherein when an indicator 41 shown in Fig. [[23]] 25 
continues arm pointing to the operation object unit 6 for a fixed period of time, the target is 
locked to the operation object unit 6 at the destination of the arm pointing. For the pointing 
action after the target has been locked, three patterns of operation are provided such as a 
crossing operation [Fig. [[23(a)]] 25(a) ], a panel touch operation [Fig. [[23(b)]] 25(b) ], and a 
panel gesture operation [Fig. [[23(c)]] 25(c) ]. 
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Please replace the paragraph at page 29, line 25 to page 30, line 17, with the following 
rewritten paragraph: 

Fig. [[24]] 26 shows a flowchart for the crossing operation. In step S151 of Fig. 
[[24]] 26, the three-dimensional direction of the arm when the indicator 41 has locked the 
target is determined, which is assumed to be the origin of the crossing operation. In step 
SI 52, it is then determined whether the arm-pointing action is continued after the target lock. 
When the arm pointing is continued (Yes), the three-dimensional direction of the arm 
pointing is obtained in step SI 53, and then it is determined in step S154 whether the direction 
of the arm pointing is within the crossing action range, wherein when it is within the range 
(Yes), a command corresponding to the direction is transmitted continuously in step SI 55. 
On the other hand, when it is determined in step SI 52 that no arm pointing is performed for a 
fixed period of time (No), the determination on arm pointing is terminated. The crossing 
operation is suitable for simple operations such as turning on/off of a switch and changing of 
the volume and channel. 

Please replace the paragraph at page 30, line 18 to page 31, line 4, with the following 
rewritten paragraph: 

Fig. [[25]] 27 shows a flowchart for the panel touch operation. In step S161 of Fig. 
[[25]] 27, the direction of the body (the direction of the shoulder) after the indicator 41 has 
locked the target is determined, which is then assumed as the base point of the panel touch 
operation. In the panel touch operation, a panel for indication (operation panel 42) is 
assumed to be present in front of the indicator 41. The operation panel 42 after the target 
lock is always set just in front of the indicator 41, as in the case of the indicator 41 in Fig. 
[[23]] 25. The indicator 41 controls the operation object unit 6 by touching part of the 
operation panel 42 (which is segmented into nine in the example of Fig. [[23]] 25). 
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Please replace the paragraph at page 31, line 22 to page 32, line 10, with the following 
rewritten paragraph: 

Fig. [[26]] 28 shows a flowchart for the panel gesture operation. In step SI 701 of Fig. 
[[26]] 28, the direction of the body (the direction of the shoulder) after the indicator 41 has 
locked the target is determined, which is assumed as the base point of the panel gesture 
operation. In the panel gesture operation, the operation panel 42 after the target lock is 
always set just in front of the indicator 41, as in the case of the indicator 41 of Fig. [[23]] 25. 
The indicator 41 controls the operation object unit 6 by the trace of arm pointing by the 
indicator 41 being determined in the range of the operation panel 42 (segmented into nine in 
the example of Fig. [[23]] 25). In other words, different operation commands can be 
generated for similar circular traces depending on the order of passage. These commands can 
be set as desired by the operating-unit registration section 35. 

Please replace the paragraph at page 33, lines 12-18, with the following rewritten 
paragraph: 

These crossing operation, panel touch operation, and panel gesture operation are 
based on making a pointing action for the operation object unit 6 after the target lock to the 
operation object unit 6 by the arm pointing. Fig. [[27]] 29 shows an embodiment in which 
the use of the display 8 shown in Figs. [[1]] 3 and [[2]] 4 allows complicated operation to be 
performed more easily. 
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Please replace the paragraph at page 33, line 19 to page 34, line 1, with the following 
rewritten paragraph: 

Fig. [[27]] 29 shows pointing-operation execution phases with a combination of the 
panel touch operation or the panel gesture operation and the display 8. In a target-lock 
execution phase 51 as the first step, the indicator 41 shows the intention of pointing the arm 
to the operation object unit 6. At that time, the indicator 41 points the arm to the operation 
object unit 6, with the display 8 present in a different direction. 

Please replace the paragraph at page 34, line 21 to page 35, line 4, with the following 
rewritten paragraph: 

Another embodiment of the panel-operation execution phase 53 using the display 8 is 
a method in which no target lock is made from the beginning. In this method, the operation 
object unit 6 [[etc.]] and the like is displayed on the display 8 as a menu, so that the indicator 
41 can control all the devices from any indoor positions while turning only towards the 
display 8. This method provides a convenient interface particularly for persons who move 
little and always face in one direction, such as persons with disabilities. 

Please replace the paragraph at page 35, lines 5-17, with the following rewritten 
paragraph: 

The operating-unit registration section 35 of Fig. [[3]] 5 allows information on the 
operation object unit 6 and various pointing actions for operating it and a combination thereof 
to be registered in the operating-unit database 34. Fig. [[22]] 24 shows a flowchart for the 
embodiment. In step SI 41, the kind [[and]] or individual information of the operation object 
unit 6 are registered first. Then, in step S142, the position of the operation object unit 6 is 
registered. In step SI 43, pointing actions and commands corresponding thereto are 
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registered. In step SI 44, as a more complicated case, combinations of pointing actions and 
whether to use a display during a pointing action are registered. A series of the operations is 
repeated necessary times. 

Please replace the paragraph at page 36, lines 10-11, with the following rewritten 
paragraph: 

(A) A noncontact Noncontact and unrestricted arm pointing actions by multiple users 
in an indoor space can be achieved. 
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